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ABSTRACT 



Communication of binary data over an additive white 
Gaussian channel by means of coded transmission over coherent 
phase-shift keyed modems is considered. It is found that 
for high rate codes and coarse receiver output quantization, 
the minimum value of E^/N o required for reliable communica- 
tion is significantly higher than the Shannon limit of 
-1.6 dB. for the raw channel. This loss of efficiency is 
investigated quantitatively as a function of code rate and 
coarseness of quantization, and shown to be negligible for 
low rate code with fine quantization. Similar results are 
presented showing the minimum E^/N q ratio for operation in 
the region where a sequential decoder performs finite 
average computation. In connection with this, tables and 
curves are given which show the value of the Pareto 
parameter p as a function of E b /N Q . 
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I. INTRODUCTION 



A. CAPACITY OF THE ADDITIVE WHITE GAUSSIAN NOISE CHANNEL 

Shannon's formula [Ref. 1] for the capacity of the band- 
limited additive white Gaussian noise (AWGN) channel, shown 
in the communication scheme of Figure 1 is 

C = i log 2 (1 + bits/baud 

o 

p 

or C = W log 2 (l + bits/second 

o 

where W is the bandwidth of channel, P is the average trans- 

N 0 

mitted power, and is the noise power spectral density. 

As W approaches infinity, i.e., the bandwidth is 
unlimited, 

w log 2 (l + jhj) 

o 



I + FT) 

o 



c = 



Lim 



Letting W = r- 



C = 



Lim 

h+0 



Lim 1 
h-»-0 h 



ln(l + Ph/N 0 ) 
In 2 
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By L'Hospital rule 



C 



Lim 

h-*-0 



P 

N + Ph 
o 

• In 2 



C 



P 

N In 2 
o 



bits/second 



This Is the capacity for the Infinite bandwidth AWGN channel 



n (t ) 




data 

stream 



output 

stream 



AWGN Channel 



Figure 1. System block diagram 

Let R denote the number of information bits to be 
transmitted per unit time (i.e., bits per second). Then, 

P 

the energy per bit is ^ . Since C is the maximum 

number of bits per second which can be transmitted with 
arbitrarily small error probability, must have reliable 
communication requires that R be less than C, or equivalently 
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R 



< C 




or 



or 




In 2 
0.69 



N q in 2 



approximately 



Expressing the ratio in decibels, this becomes 




> - 1.6 



This is called the "Shannon limit." 

The significance of this limit is that no matter what is 
in the boxes labeled "modulator" and "demodulator" in Figure 
1, the probability of error can be made arbitrarily small 
only if E b /N Q exceeds -1.6 dB; conversely, if E b /N 0 does 
exceed -1.6 dB, then there exists some "modulator" and 
"demodulator" to make the probability of error as small as 
desired. 

B. DISCRETIZING THE CHANNEL 

1. Bit-by-bit Antipodal PSK 

u ' ' ^ ‘ - ' ' f 

One modulation for binary communication over the 
AWGN channel is the antipodal PSK system shown in Figure 2. 
The data stream is a sequence of ±l’s so that over each 
baud of length T, the transmitted signal s(t) is a cosine of 
frequency to and phase either 0 or it. Note that the energy 
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expended for information bit is 



/ [s(t)]‘ 

0 



dt = 



/ 

0 



2E 



b n 2 .. 

— cos a) J dt 
T c 



= E, 



where the simplifying assumption is made that a baud contains 
an integral number of cycles of the cosine. The demodulator 
in this system is a correlation detector which requires an 
oscillator synchronized with that at the transmitter; the 
system is thus called a coherent system. The integrator 
output at time T is tested against the threshold 0. If 
Z(T) .> 0, the receiver decides that the data bit is +1, 
if Z(T) < 0, the decision is -1. The system operates 
identically over each baud, independent of all other bauds, 
and is therefore called a bit-by-bit system. 




0 < t < T 



Figure 2. A model of bit-by-bit antipodal PSK system 
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Letting R again denote the data rate, the baud length 
must satisfy T = ^ . 

Let the power spectral density function of white 
N 

noise be j thus its autocorrelation function is 



R(x) 






Consider in the case of signal input alone. 



y(t) 




cos w c t 



from the integrator output at time T, 



±2 /eT T 9 

Z (T) = m “ / cos^ to t dt 

s x Q c 

- , assuming that 

[0,T] contained integral number of cycles of the cosine. 

In the case of noise present alone, the output is 



Z n (T) 




n(t) cos to t dt 
c 



So, when noise and signal are presented in the system 
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y(t) = s(t) + n(t ) 



Z(T) = Z s (T) + Z n (T) 



, T rp 

= ± j E fc ' + / J ~ n(t) cos ca c t dt 



The quantity Z n (T) is a zero mean Gaussian random variable 

2 

whose variance, o z , can be found as follows: 



n 



E[(Z n (T)r] 



= E[ 



T 



/ J £ n(t) cos w c t dt) 



( T 

E[j| |/ n(t) cos w c t dt 
T T 



0 



n(s) cos w c s ds 



% / / E 
1 0 0 



n(t)*n(s)| cos w t*cos w s dt ds 

c c 



But 



R(t) = E[n(t ) • n(t-x ) ] 



n 



T T 



/ / R(t-s) cos w t*cos to s dt ds 



0 0 
T T 



N 



J / 6(t-s) cos a) t*cos w s dt ds 



T 6 6 2 

f / COS 2 (0 c t dt 

N 
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The conditional probability density function of Z(T) given 
the data bit can be represented by Figure 3» 




Figure 3. Conditional probability density function of 
Z(T) in the case of a positive input of 
amplitude 1, and in that of a negative input 
of amplitude -1. 



Then, the probability of error, Pe, of the output 
stream after the threshold device can be found by- 



Pe = Pr [ ( Z (T) J> 0) | data bit = -1] - Pr.Cdata bit = -1] 
+ Pr[(Z(T) < 0) | data bit = +1] • Pr[data bit = +1] 

Assuming equally likely data, _ | 

Pr[data = -1] = Pr[data = +1] = ^ 
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By symmetry 



Pr[Z(T) > 0 | data bit = -1] 
thus 

Pe = Pr[(Z(T) ^ 0) | data 
or Pe = Pr[(Z(T) < 0) | data 



= Pr [ ( Z (T) < 0 | data bit = +1] 

bit = -1] 
bit = +1] 



Since this conditional probability density function is 

2 N 0 

Gaussian with zero mean and variance a = 



Pe 



1 

2 



1 

2 



/&b p ? 

1 / e -(x 2 /2a 2 ) 

y 2ir a 0 



dx 



— J 
J 2tt 0 



/E^/a 



e" (y /2) dy , 



1 

2 



1 

yi^ 6 



where y 

. 2 .., 

/ e y ^ dy 



x 

a 




) 



The probability of error is shown by the shaded area 
of the standard normal distribution function in Figure 4, 
where 



f (y) 



i e -(y 2 /2) 
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In Figure 5 the curve of probability of error versus 
E b /N o in decibel is shown, along with the Shannon limit. 



f(y) 




Figure Probability of error. Pe = Q( 




10 

10 

10 

10 

10 




Figure 5> Probability of error as a function of SNR. 
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_5 

It can be seen that, for example, to obtain Pe of 10 
with this scheme, (E^/N^^g must be 9.6 dB. On the other 
hand, the capacity theorem says that there is some system 
with essentially zero error probability whenever E^/N 0 
exceed -1.6 dB [Ref. 2]. Thus, there is a potential gain of 
11.2 dB over the antipodal PSK system. 

2 . Antipodal PSK with Rate 1/2 Coding 

Part of the potential gain indicated in the previous 
section can be realized when the somewhat more complicated 
system of Figure 6 is used. This system still employs a 
coherent antipodal PSK modulation/demodulation scheme, but 
it differs from the previous system (Figure 5) in two impor- 
tant respects: (1) more bits are transmitted per unit time 
than appear as data bits (in this case, twice as many); and 
(2) the bits transmitted at any time depend not only on the 
present data bit, but also on some past data bits (thus this 
is not a bit-by-bit system) . 




Figure 6. A model of antipodal PSK with rate 1/2 coding 
system. 



Let data stream input be 1 bit for each T second interval, 

then, for code rate, R = 1/2, (R is defined to be the number 

of information bits per transmitted bit), the output from 

encoder will be 2 bits for each T second. 

Let E be the energy expended per transmitted bit then 
s 

E = (1/2)E, . Assume that the use of the channel is as fol- 
s b 

lows: each bit is transmitted by binary antipodal signaling. 

At the receiver, the best guess is made for each bit and +1 
or -1 is the output, accordingly. Thus, from the output of 
the encoder to the input of the decoder, the channel can be 
though of as a discrete system with binary input and binary 
output acting as illustrated graphically below, where p is 
the error probability for bit-by-bit binary antipodal signal- 
ing. According to the solution in bit-by-bit antipodal PSK 

, 2 E s 

found previously, p = Q( — ) 




This discrete channel, which is called the binary 
symmetric channel, has a capacity, C, with the following 
significance: if the code rate R satisfied R < C, then there 

exists a code (i.e., an encoder-decoder pair) such that the 
error probability can be made as small as desired. 
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The capacity of the binary symmetric channel is 

given by 

C = 1 - H(p) bit/use of channel, 

where H(p) = -p log 2 p - (l-p)log 2 (1-p) 




Thus for reliable communication in this system, it is 
necessary that 



R = 



< C = 1 - H 



Q( 




= 1 - H 



Q( 



'2-E b /2 

IT 



or 



N, 



dB 



> 1.8 dB. 



When this condition is satisfied, there is some choice of 
encoder and decoder to make Pe. as small as desired. 

Note that this minimum E./N is still 3.^ dB above 

b o 

the Shannon limit. This difference is due to use of R = 1/2 
rather than a lower rate, and making hard decisions at the 
integrator output. In Chapter II, the minimum E b /N Q is 
determined for various rates and output quantizations, and 
it is shown that for low-rate codes and fine output 
quantization, the Shannon limit is approached. 
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1 




V N o ’ dB 

Figure 7 . Minimum E^/N , in dB, due to R = 1/2 

C. CAPACITY OF DISCRETE CHANNELS 

Consider a discrete memoryless channel (DMC) whose input 
alphabet X consists of the K integers, 0, 1, 2, ..., K-l, 
and whose output alphabet Y consists of the J integers, 

0, 1, 2, ..., J-l. Using integers for input and output 
letters simplifies the notation somewhat in what follows and 
also reemphasizes the fact that the name given to input and 
output letters are of no concern whatsoever. 

The channel is specified .by a transition probability 
assignment, P(j|k), given for 0 = j = J-l, and 0 = k = K-l, 
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by definition, P(j|k) is the probability of receiving integer 
j given that integer k is the channel input. For example, 
see Figure 8. 

Since the channel is memoryless, each output letter in 
the sequence depended only on the corresponding input, and 
the probability of an output sequence Y = (y^, y 2 , ••• y^) 
given an input sequence X = (x^, x 2 , ••• x^) is given by 



Let Q(k) be the probability of using integer k, then the 
average mutual information [Ref. 3] between the input and 
output is 



where, the probability of receiving integer j is written as 




K-l J-l 

I(X;Y) = l l Q(k) • P ( j | k ) log 
k=0 j =0 



2 K-l 



l Q(k)P(j|i) 
i=0 



P(,1 Ik) 



l Q(i)P( j | i) 



P(j | k ) 



i 




0 



1 



2 



Y k 



j X 



- J-2 



J-l 



Figure 8. Transition probability for K = 2 input 



and J output 
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The capacity of C of a discrete memoryless channel (DMC) 
is defined as the largest average mutual information I(X;Y) 
that can be transmitted over the channel in one use, 
maximized over all input probability assignments. 



C 



Q(0) 



max l Q(k) P(j |k) log 2 

. . . Q(K-l). k,j 



PU I k ) 

"TOT" 



C is the function only of the channel. The calculation 
of C involves a maximization over K variable with both ine- 
qualities constrains, Q(k) = 0 and an equality constrains, 

£ Q(k) = 1. The maximum value must exist since the function 
is continuous and the maximization is over a closed bounded 
region of vector space. 

The channels of interest here all have two inputs and 
are symmetric, as shown in Figure 9> so that the capacity 
formula reduces to 



C 



1 

2 



J-l 

l 



3“0 



[p(j|o) iog 2 + PUli) 



log 2 



pq Id 

"TOT 



] 



p(j|0) = P(J-j-l|l) = A J+1 



Then 



0 ■ + + --- +A 3 l0 S 2u J+ A l)/2 



J 

l 

i=l 



C = 1.0 + l A. log 0 — — 

<V A j-i+i> 



where A^, i=l,2,...3» is the conditional probability of receiving 
output letter j given that input letter k is transmitted. All 
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details of modulator, channel and demodulator are compressed 
into the probabilities {A^} 




J-2 

J-l 



P(j=0) = |(A 1 + A ) 



P(j=l) » J(A 2 + A J _ 1 ) 



P(j=J-l) = i(Aj + A 1 ) 



Figure 9. Conditional probabilities P(j|k) 
for J output and K = 2 input. 



P(j 1 0) = P(J-J*-1 1 1) = A j+1 



The amplitude quantizer may be one of the five shown in 
Figure 10, denoted J = 2,3>^ } 8 and l6[Ref. A], Considering 
for J = 2, K = 2 



J "i 

C = 1.0 + l A^ log 2 



i=l 



A. +A t 
l J-i+1 



= 1.0 + A^ log 2 A^ + A 2 log 2 A 2 



= 1.0 + plogp p + (1-p) iog 2 (i-p) 



where p = 1-p = 

thus C = 1.0 - H(p), 




channel capacity. 
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(e) 



15 


in 


13 


12 
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10 


9 


8 
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6 


5 


4 


3 


2 


1 


A l6 


*15 


*14 


A 13 


CM 


A 11 
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a 8 


A 7 


A 6 


A 5 


A 4 


A 3 


A 2 



-7a -6a -4a -2a a 0 a 2a 4 a 6a 7ct 




Fig. 10. Amplitude quantizers (a) J=2, hard 

decision, (b) J=3> binary- symmetric 

erasure, (c) J=4 with double null zone, 

(d) J=8, (e), J=10 with a= t /4 . 

o 
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D. SEQUENTIAL DECODING 

A sequential decoder is one which decodes by making 
tentative hypothesis as it goes along and by changing these 
hypothesis, when subsequent choices indicate an earlier 
incorrect hypothesis [Ref. 3, page 263 ]. 

From Gallager [Ref. 3], a property of sequential de- 
coding systems is that the computation C required to decode 
a digit is a Pareto random variable; that is, its distribution, 
for large L assures the form 

Pr[C > L] = KL~ P as L approaches 00 



where p is the solution of R = R. 



E o (p) 

R = -2 

P 



bits/channel use 



J—l K~ 1 

E (p) = Max { — logp l [ l Q(k ) P( j | k) 1/1+P ] 1+p } 
Q(k) j = 0 k=0 



0 < p < 



where the general formula of E Q (p) is shown in Appendix A. 

The computation a sequential decoder must perform to 

decode an information digit is a random variable, and Rpis 

the rate above which the p^* 1 moment of this random variable 

fails to exist. when p = 1, the symbol R is customary 

comp 

[Ref. 5]. In order to operate with finite average 
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computation. R must be less than R . This condition can 
^ 5 comp 

be translated into a minimum value of E. /N in the same way 

bo 

that the condition of R < C yield a minimum value of E^/N^ 
In Chapter III, minimum values of E b /N o are given for various 
rate and quantisation levels. 
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II. CAPACITY LOSSES DUE TO CODING AND OUTPUT QUANTIZATION 



From page 19 , the channel capacity is 



C = 1.0 + 



J 

I A ± 

i=l 1 



log. 



A. 



A i +A J-i+l 



Curves for various amplitude quantizers as shown in Figure 10 
are presented in Figure 11. The curves are plotted for the 
value of E^/N q required to achieve C = R versus R for 
J = 2,3, 4,8 and 16, where K = 2. 

Table I gives the value of E b /N Q vs R for various 
values of J. 
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III. R LOSSES DUE TO CODING AND OUTPUT QUANTIZATION 
comp 

Curves of E b /N o versus R are presented in Figure 12 for 

various values of J to achieve p = 1, or R = R. where 

* comp * 

K = 2. 

Table II gives the value of E^/N q vs R. Here, as in 
Table I, R is treated as a continuous variable, although 
in practice R is usually the reciprocal of integer to 
simplify coder and decoder design. 
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IV. PERFORMANCE OF p VERSUS E. /N FOR R = R 

D O p 

E (p) 

By solving R = R p = — - — , curves of p vs E^/N q 
for various values of R are presented in Figures 13 > 14 and 
15 for J =2,4 and 8 respectively and are useful for design 
tradeoff. 

It will be seen that, for example, in Figure 15 at 
p = 1 the value of E^/N o is 2.65 dB for R = 1/2 corresponded 
to Figure 12 at R = 1/2, and at p = 0 the value of E b /N Q is 
-0.66 dB for R = 1/4 that is the same with the value of 
E b /N Q in Figure 11 for J = 8, R = 1/4. 
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TABLE I 



Table 


of <E b /N 0 


)dB Required to 


Achieve C 


= R 


R 


J=2 


J=3 


J=4 


J=8 


J=l6 


3/4 


3.01 


2.34 


2.17 


1.76 


1.66 


2/3 


2.51 


1.75 


1.56 


1.18 


1.09 


1/2 


1.77 


0.90 


0.65 


-.30 


0.22 


1/3 


1.21 


0.28 


-0.04 


-.37 


-.45 


1/4 


0.97 


0.02 


-.32 


-.66 


-.74 


1/8 


0.65 


-.32 


-.70 


-1.04 


-1.13 


1/16 


0.51 


-.47 


-.88 


-1.22 


-1.31 


1/32 


0.44 


-.55 


--.96 


-1.30 


-1.40 


1/64 


0.40 


-.58 


-1.00 


-1.35 


-1.44 


1/128 


0.39 


-.60 


-1.02 


-1.37 


-1.46 



TABLE II 

(E b /N Q )dB Required to Achieve p 


=1, R = R 

' c oTnp 


R 


J=2 


J=3 


J=4 


- J=8 


J=l6 


3/4 


5.71 


4.54 


4.37 


3.67 


3.52 


2/3 


5.26 


4.10 


3.85 


3.25'- ' 


3.11 


1/2 


4.59 


3.46 


3.18 


2.64. 


2.51 


1/3 


4.10 


2.99 


2.70- 


2.20 


2.08 


1/4 


3.89 


2.80 


2.51 


.2.02 


1.90 


1/8 


3. 62 


2.55 


2.25 


1.78 


1.67 


1/16 


3.50 


2.44 


2.14 


1.6a 


1.56 


1/32 


3. 44 


2.39 


2.09 ■ 


1.63 . 


1.51 


1/64 


3.41 


2.36 


2.06 


•1.60 


1.49 


1/128 


3. 39 


2.35 


2.05 


1.59. 


1.48 
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Figure 11. E b /N required to achieve 
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Figure 12. E h /N required to achieve 
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Figure 13. Dependence of p on E. /N , dB. 
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Figure 14. Dependence of p on E./N , dB. 
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Figure 15. Dependence of p on E^/N , dB. 



V. SUMMARY AND CONCLUSION 



The objective has been to present a unified treatment 
of efficiency losses due to coding and quantization in a 
PSK system with additive white noise. Performance curves 
and tables are given for useful design tradeoffs. 

In Chapter I the basic concept of the Shannon limit with 
respect to antipodal bit-by-bit PSK is presented. In 
Chapter II,. Ill, and IV, tables and curves showing expected 
system performance are presented. The expected performance 
figures were obtained by computer-aided analysis. 

It is shown that in the limit of fine output quantization 
and low-rate coding the Shannon limit is approached. 
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APPENDIX A 



GENERAL FORMULA OF E (p) AND R 

o K comp 



From Gallager [Ref. 3] define for a discrete memoryless 
channel with K input symbols and J output symbols. 



E C (P) 



M S X {- log, V [ V Q(k)P(j|k) 1/1+p ] 1+p } 
w j =0 k=0 



and R. 



E 0 ( P) 



Let 1+p = r , QOO max = Q(k) , K = 2 

E ( p ) = - logp(Q(k) ) r - logp J f[P(J|0) 1/r +P(j|l) 1/r ] r 

° ^ j = 0 

= - log 2 (Q(k)) r - log 2 {[P(0|0) 1/r +P(0|l) 1/r ] r 

[P(l| 0) 1/r +P(l| l) 1/r ] r 



+ [P(J-l|0) 1/r +P(J-l| l) 1/r ] r } 
E o (p) = - l0 g 2 (Q(k)) r - log 2 {(A]; /r + Aj /r ) r + (Al /r + A^) r + 

+ ( Aj /r> + A^ r ) r ) 



where A^, i = 1,2,3,...,J is the conditional probability, 
of J given k as shown in the following figure for K= 2. 
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0 




2 

j 

J-2 

J-l 



A j+1 = P(J 1 0) 

= P(J-j-l|l) 



E 0 (P) 



- - 1o S 2 'ma k x f - log 2 C 



For Q(k) maximum = ^ 



R 



comp 



J 

-2.0 log 2 (|) - log 2 CJ i (aJ /2 +aJ^ +1 ) 2 ] 

2.0 - log2[(A^ /2 +Aj /2 ) 2 + (A2 /2 +A^ 2 ) 2 + ... 



R 



comp 



+ (a} /2 + a! /2 ) 2 ] 



r i j y ti A ) 1/2 1 

2.0 - log, [2 l A, + 2 l K r± J-i+l' J 

^ i=l 1 i=l i 



1.0 - log 2 [1.0 + J i (A i A J _.^) 1/2 ] 
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